LANE RECOGNITION SYSTEM FOR VEHICLE 

BACKGROUND OF THE INVENTION 

The present invention relates to a lane 
recognition system which recognizes lane makers on a 
traveling road of a vehicle. 

Japanese Patent Provisional Publication No. 
5-314396 discloses a lane-marker recognizing and 
tracking system in which a plurality of lane -marker 
detecting areas are defined in an image showing a 
road ahead of a vehicle in order to detect lane 
markers . 

SUMMARY OF THE INVENTION 

However, this system yet has a problem to be 
solved in order to further accurately and quickly 
detect the lane markers . 

It is therefore an object of the present 
invention to provide a lane recognition system which 
is capable of executing a lane recognition process 
while maintaining a detection accuracy and 
shortening a processing time. 

An aspect of the present invention resides in a 
lane recognition system which is for a vehicle and 
comprises a camera set, a memory and a controller. 
The camera set picks up a road image of a road ahead 
of the vehicle. The memory stores parameters 
representative of a model lane marker of the road 
image. The controller is coupled to the camera set 
and the memory. The controller is arranged to set a 
plurality of lane-marker candidate-point detecting 
areas on the road image so that each one of the 
lane-marker candidate-point detecting areas is 
partially overlapped with the lane -marker 
candidate-point detecting areas adjacent to the one 
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of the lane-marker candidate-point detecting areas, 
to detect each lane marker candidate point in each 
lane-marker candidate-point detecting area, to 
calculate variations of the parameters by comparing 
5 the lane marker candidate points with corresponding 
points on the model lane marker derived from the 
parameters stored in the memory, to correct the 
parameters according to the variation, and to output 
road- shape indicative information on the basis of 

10 the parameters . 

Another aspect of the present invention resides 
in a method for recognizing a lane traveled by a 
vehicle. The method comprises a step for picking up 
a road image of a road ahead of the vehicle; a step 

15 for storing parameters of a model lane marker of the 
road image; a step for setting a plurality of 
lane-marker candidate-point detecting areas on the 
road image so that each one of the lane -marker 
candidate-point detecting areas is partially 

2 0 overlapped with the lane -marker candidate -point 
detecting areas adjacent to the one of the 
lane-marker candidate-point detecting areas; a step 
for detecting each lane marker candidate point in 
each lane-marker candidate-point detecting area; a 

25 step for calculating variations of the parameters by 
comparing the lane marker candidate points with 
corresponding points on the model lane marker 
derived from the parameters stored in a memory; a 
step for correcting the parameters according to the 

30 variation; and a step for outputting road- shape 
indicative information on the basis of the 
parameters . 

BRIEF DESCRIPTION OF THE DRAWINGS 



Fig. 1 is a schematic view showing a first 
embodiment of a lane recognition system according to 
the present invention. 

Fig . 2 is an explanatory view showing an 
installation position of a camera employed in the 
lane recognition system of Fig. 1. 

Fig. 3 is a flowchart showing a lane marker 
detecting process employed in the embodiment 
according to the present invention. 

Fig. 4 is a view showing a vanishing point 10, 
model lane markers 12 and lane -marker 
candidate-point detecting areas 11 in an image 
picked up by the camera. 
DETAILED DESCRIPTION OF THE INVENTION 

Referring to Figs. 1 to 4, there is shown an 
embodiment of a lane recognition system S according 
to the present invention. 

As shown in Fig. 1, the lane recognition system 
S is installed to a vehicle VE, and comprises a 
camera 1, an image processor 2, a controller 3, a 
memory 4, a sensor unit 5, a vehicle control 
apparatus 6, an alarm device 7 and a display 8. 

Controller 3 is basically constituted by a 
microcomputer, and is coupled to image processor 2, 
memory 4 and sensor unit 5 so as to interactively 
communicate with each other. Controller 3 is 
further coupled to vehicle control apparatus 6 , 
alarm device 7 and display 8 so as to output command 
signals thereto, respectively, and is further 
coupled to camera 1 through image processor 2. 

Camera 1 is installed in a passenger 
compartment of vehicle VE. More specifically, 
camera 1 is installed at an upper and laterally 
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center position of a front window in a passenger 
compartment as shown in Fig. 2 so that a pan angle a 
between an optical axis of a lens of camera 1 and a 
longitudinal center axis of vehicle VE is zero and a 
5 tilt angle (3 of camera 1 is p. Camera 1 picks up an 
image of a road view ahead of vehicle VE . Image 
processor 2 is coupled to camera 1 and receives data 
of the image picked up by camera 1 . Image processor 
2 processes the image in order to detect lane 

10 markers of a traveling lane and sends the processed 
image data to controller 3 . 

Controller 3 transforms a shape of lane markers 
into a mathematical model by using a plurality of 
parameters representative of a road shape and a 

15 vehicle behavior. By updating the parameters so as 
to correspond the detection result of the lane 
markers with model lane markers, controller 3 
detects the actual lane markers and recognizes the 
road shape ahead of vehicle VE. Further, controller 

20 3 corrects a position of a vanishing point 9 at 

which the two lane markers cross with each other, on 
the basis of the yaw angle and the pitch angle of 
vehicle VE relative to the road. 

Memory 4 is a storage device for storing the 

25 parameters of the road model and the like. Sensor 
unit 5 detects a vehicle speed and a steering angle 
of vehicle VE . Vehicle control apparatus 6 executes 
a steering control, an acceleration control and a 
brake control according to command signals "outputted 

30 from controller 3. 

In this lane recognition system S according to 
the present invention, as shown in Fig. 5, camera 1 
and image processor 2 functions as an image pickup 
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means . Memory functions as a road shape storing 
means . Controller 3 functions as a coordinate 
transforming means, a lane-marker candidate-point 
detecting area setting means, a lane-marker 
5 candidate-point detecting means, a parameter 

variation calculating means, a parameter updating 
means and an output signal selecting means . 

A flowchart of Fig. 3 shows a procedure of a 
lane marker detecting process executed by controller 
10 3. 

At step SI, controller 3 initializes the 
parameters representative of the road shape and the 
vehicle behavior. Hereinafter, these parameters are 
called road parameters . A road coordinate system 

15 representing an actual space ahead of the vehicle is 
transformed into an x-y image -plane coordinate 
system shown in Fig. 4. In this x-y image -plane 
coordinate system, model lane markers 12 are 
represented by the following equation (1) using the 

20 road parameters . 

x = (a+ie) (y-d) + b / {y-d) + c (1) 

where a, b, c, d and e are the road parameters, and 
i is 1 and 0 . Assuming that a vertical dimension 
between camera 1 and a road surface is constant, 

25 road parameter a denotes a lateral displacement of 
vehicle VE between the lane markers , b denotes a 
road curvature, c denotes a yaw angle of vehicle VE 
(the optical axis of camera 1) relative to. the road, 
d denotes a pitch angle of vehicle VE (the optical 

3 0 axis of camera 1) relative to the road, and e 
denotes a lane width between the lane markers . 
Further, c includes a pan angle a within an . 
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installation angle between the host vehicle and 
camera 1, and d includes a tilt angle (3 within an 
installation angle between the host vehicle and 
camera 1 . 

5 Under the initial condition, the shape of the 

road and the lane markers and the vehicle behavior 
are set at values corresponding to center values, 
respectively, since the shape of the road and the 
lane markers and the vehicle behavior are not clear 

10 in this initial condition. More specifically, the 

lateral displacement a of vehicle VE within the lane 
markers is set at a center between the lane markers , 
the road curvature b is set at zero (straight), the 
yaw angle c relative to the lane markers is set at 

15 pan angle a, the pitch angle d relative to the lane 
markers is set at tilt angle p* under the vehicle stop 
condition, and lane width e between the lane markers 
is set at a lane width of a highway representatively 
defined by the rule of a road structure. 

20 Further, the road parameters may be initialized 

on the basis of values representative of the vehicle 
behavior detected by sensor unit 5. More 
specifically, when the steering wheel is being 
turned to right or left under the initial condition, 

25 it may be determined that vehicle VE travels on a 
curved road of a radius corresponding to the 
steering angle, and therefore the road curvature b 
may be set at a value corresponding to the steering 
angle. 

30 At step S2, controller 3 initializes small 

areas for detecting a candidate lane marker as shown 
in Fig. 4. Under the initial condition, since it is 
supposed that there is a large difference between 



the model lane markers obtained by inputting the 
initial values into the respective road parameters a 
to e and the actual lane markers on the image plane, 
it is preferable that the lane-marker 
candidate-point detecting areas are set as large as 
possible. As shown in Fig. 4, in this embodiment 
twelve detecting areas including six right detecting 
areas and six left detecting areas are set to detect 
right and left lane markers 9 . If lane markers 9 
have been detected already in the previous process, 
the size of each the lane-marker candidate-point 
detecting area is set small so as to lower the 
possibility of erroneous detection of other object 
as a lane marker. Further this small setting 
improves the processing speed of this process. 

At step S3 , controller 3 receives the image 
data which image processor 2 obtains by processing 
the image picked up by camera 1 . 

At step S4 , controller 3 sets the lane-marker 
candidate-point detecting areas on the road image 
received from image processor 2. During this 
setting, the lane-marker candidate-point detecting 
areas are set so that the model lane markers are 
located at centers of the respective lane -marker 
candidate-point detecting areas, on the basis of the 
lane-marker candidate-point detecting areas 
calculated at step S2 and the model lane markers 
obtained from the road parameters calculated at step 
SI or steps S12, S14 and S15. It will be" understood 
that the lane -marker candidate -point detecting areas 
may be set at positions offset from the model lane 
markers according to the change of the past model 
lane markers. More specifically, in order to 



correspond the y- coordinate of a start point of each 
lane-marker candidate-point detecting area with the 
y-coordinate of an end point of the previous 
lane-marker candidate-point detecting area, each 
lane-marker candidate-point detecting area is set so 
that the y-coordinate of each lane-marker 
candidate-point detecting area is overlapped with 
the y-coordinate of the adjacent lane-marker 
candidate-point detecting area by a line. 

At step S5, controller 3 detects the lane 
marker candidate point in each lane-marker 
candidate-point detecting area. In this detecting 
operation, a differential image is produced by 
filtering the input image with the Sobel filter. 
Then, controller 3 counts suitable pixels of each 
line segment connecting a point on an upper base 
line and a point on a lower base line of each 
detecting area. The suitable pixel is a pixel which 
is located on the line segment and whose density is 
greater than a predetermined value capable of 
extracting the detection line. Further, a line 
segment, which has the largest number of the 
suitable pixels, is selected from all line segments 
in each detecting area and is determined as a 
detected straight line. The coordinate of the start 
point of the detected straight line is outputted as 
an output value of the lane marker candidate point. 

When the number of the suitable pixels of the 
determined detection straight line is smaller than a 
predetermined rate of the maximum number of pixels 
corresponding to the length of the detecting area, 
controller 3 determines that there is no lane marker 
candidate point in this detecting area. 



For example, under a condition that the number 
of pixels corresponding to the length of the 
detecting area is fifteen and the predetermined rate 
is 1/2, if the number of the suitable pixels of the 
detection straight line are seven or less, 
controller 3 determines that there is no lane marker 
candidate point in this detection area. If the 
number of the suitable pixels of the detection 
straight line segment is nine, controller 3 
determines that there is a lane marker candidate 
point in this detecting area. Controller 3 treats 
the coordinate of the start of the selected line 
segment as the line marker candidate points in this 
detecting area and stores the coordinate of the end 
of the selected line segment is stored. The start 
point of the selected line segment corresponds to a 
point of the upper base line of the line-marker 
candidate-point detecting area, and the end point of 
the selected line segment corresponds to a point of 
the lower base line of the line-marker 
candidate-point detecting area. 

The above operation of determining the 
lane -marker candidate points is executed by each 
lane -marker candidate-point detecting area from the 
far side to the near side of the road image in turn. 

In determining the lane-marker candidate points, 
the predetermined rate relative to the length of the 
lane-marker candidate -point detecting area may be 
set at a constant rate throughout all detecting 
areas or may be varied by each detecting area. 
Further the predetermined value of the density may 
be set at a constant value throughout all detecting 
areas or may be varied by each detecting area. 
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When the lane -marker candidate point is not 
detected by the execution of the above-discussed 
processing, the detection result of the end point of 
the previous lane marker candidate point detecting 
area is outputted as the start point of the present 
area in which the lane marker candidate point is not 
detected. 

Although the coordinate at the point on the 
upper base line in the lane-marker candidate-point 
detecting area is employed as the output result of 
the lane marker candidate point and the processing 
of each lane-marker candidate-point detecting area 
is executed in turn from the far side to the near 
side, it will be understood that the present 
invention is not limited to this. Instead of the 
coordinate of the upper base line, the coordinate of 
the lower base line of each lane -marker 
candidate-point detecting area may be employed as an 
output result of the lane-marker candidate point. 
Further, the processing order of the lane-marker 
candidate-point detecting areas may be changed so 
that the operation is executed from the near side to 
the far side. 

At step S6, controller 3 checks whether or not 
the number of the lane -marker candidate points of 
the whole lane-marker candidate-point detecting area 
is greater than or equal to a predetermined value 
agreeable to decide the line obtained from the 
lane-marker candidate points as a lane marker. When 
the number of the lane marker candidate points is 
smaller than the predetermined value, controller 3 
determines that there is no lane marker in the 
detecting areas, and the routine of this flowchart 
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returns to step S2 to again initialize the size of 
the detecting area. When the number of the 
lane-marker candidate points is greater than or 
equal to the predetermined value, the routine 
proceeds to step S7. 

At step S7, controller 3 calculates an offset 
quantity between the determined lane-marker 
candidate point and a point on the model lane marker 
obtained by the previous processing by each 
lane-marker candidate point. 

At step S8, controller 3 calculates variations 
Aa, Ab, Ac, Ad and Ae of the road parameters a to e 
on the basis of the respective offset quantities of 
the lane-marker candidate points. The calculation 
of the variations Aa to Ae may be executed on the 
basis of a least -square method, for example, 
disclosed in Japanese Patent Provisional Publication 
No. 8-5388. 

At step S9, controller 3 corrects road 
parameters a to e on the basis of variations Aa to 
Ae calculated at step S8. When the model lane 
marker expressed by the equation (1) is employed, 
the correction of road parameters a to e is executed 
by using the following equations (2). 
a = a + Aa, b = b + Ab , c = c + Ac, 

d = d + Ad, e = e + Ae (2) 

At step S10 following the execution of step S9, 
controller 3 determines whether or not an- estimation 
of the road shape is normal. More specifically, 
controller 3 determines whether or not the road 
parameter representative of the road shape is normal. 
When the determination at step S10 is negative, the 
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routine proceeds to step S14 wherein controller 3 
initializes the road parameter representative of the 
road shape . In the equation ( 1 ) , parameter b 
represents the road curvature, and parameter e 
5 represents the road width. Accordingly, when it is 
determined on the basis of the vehicle behavior 
indicative value detected by sensor unit 5 that the 
road curvature estimated from parameter b is never 
generated at the traveling road, parameter b is 

10 initialized. Similarly, when it is determined on 
the basis of the vehicle behavior indicative value 
detected by sensor unit 5 that the road width 
estimated from parameter e is never generated on the 
traveling road, parameter e is initialized. 

15 When the determination at step S10 is 

affirmative, the routine proceeds to step Sll 
wherein it is determined whether or not an 
estimation of the vehicle behavior is normal. That 
is, it is determined at step Sll whether or not the 

20 road parameter representative of the vehicle 
behavior is normal. 

When the determination at step Sll is negative, 
the routine proceeds to step SI 5 wherein controller 
3 initializes the road parameter representative of 

25 the vehicle behavior. In the equation (1) 

representative of the lane marker model, parameter a 
represents the lateral displacement within the lane, 
parameter c represents the yaw angle relative to the 
road surface, parameter d represents the pitch angle 

30 relative to the road surface. Accordingly, when it 
is determined on the basis of the vehicle behavior 
indicative value detected by sensor unit 5 that the 
lateral displacement estimated from parameter a is 
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never generated on the actual traveling road, 
parameter a is initialized. Similarly, when it is 
determined on the basis of the vehicle behavior 
indicative value detected by sensor unit 5 that the 
yaw angle estimated from parameter c is never 
generated on the actual traveling road, parameter c 
is initialized. Further, when it is determined on 
the basis of the vehicle behavior indicative value 
detected by sensor unit 5 that the pitch angle 
estimated from parameter d is never generated on the 
actual traveling road, parameter d is initialized. 

At step S12 following the affirmative 
determination at step Sll, controller 3 stores the 
road parameters a to e corrected at step S9 as road 
parameters of the lane marker model in memory 4 . 

At step S13, controller 3 estimates the road 
shape from the new road parameters, and outputs the 
command signal according to the estimated road shape 
to vehicle control apparatus 6, alarm device 7 and 
display 8 . 

Following the execution of step S13, S14 or S15, 
the routine returns to step S3. 

With the thus arranged lane recognition system 
S according to the present invention, the 
lane-marker candidate-point detecting area is set so 
that a part of the lane -marker candidate -point 
detecting area is overlapped with the adjacent upper 
and lower lane-marker candidate-point detecting 
areas. This enables the road information" to be 
commonly and continuously utilized, and this 
facilitates the detection of the lane marker 
candidate points in the continuous lane-marker 
candidate-point detecting areas. 
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Further, since the overlapped arrangement of 
the detecting areas facilitates the comparison 
between the two detection results of the adjacent 
detecting areas, the detection accuracy of a curved 
road is improved. 

Furthermore, the present invention is arranged 
such that the coordinates of the overlapped other 
detecting area is employed as a detection result of 
the objective detecting area when no candidate point 
is detected at the objective detecting area. This 
arrangement enables an interpolation during the no 
candidate point condition to be facilitated. 
Further, this arrangement improves the detection 
accuracy of the lane marker during the traveling on 
a curved road as compared with the general 
interpolation. 

This application is based on a prior Japanese 
Patent Application No. 2000-395096 filed on December 
26, 2000 in Japan. The entire contents of this 
Japanese Patent Application are hereby incorporated 
by reference. 

Although the invention has been described above 
by reference to a certain embodiment of the 
invention, the invention is not limited to the 
embodiments described above. Modifications and 
variations of the embodiment described above will 
occur to those skilled in the art, in light of the 
above teaching. The scope of the invention is 
defined with reference to the following claims. 



